Query Term Selection Strategies for Web-based Chinese Factoid Question Answering

Passage retrieval plays an important role in a Chinese factoid Question Answering (QA) system. Query term selection is the process of choosing keywords from a given question to make the most use of information retrieval engines. Query terms selected by humans are analyzed to measure the difficulty and for evaluating machine generated results. Three approaches, namely stop words elimination, rule-based, and machine learning-based, are studied in this paper. Eliminating stop words is the simplest one. Heuristic rules produced by morphologists are more complex. Conditional Random Fields (CRF), a machine learning approach, is adopted for labeling query terms. For evaluation, two sets of metrics are proposed. Passage MRR/Coverage relies on search engine result which directly relates to the QA performance but is time consuming and may vary at different time. Our experiment shows that Query Term Precision/Recall is a viable alternative. The baseline Coverage of sending raw questions to Google is about 53%, while applying the three approaches yields 65% for stop words elimination, 57% for rule-based approach, and 54% for machine learning-based approach. The MRR of sending raw questions to Google is 0.33, while applying the three approaches yields 0.44 for stop words elimination, 0.41 for rule-based approach and 0.38 for machine learning-based approach. The result can be not only for factoid QA systems but also a preprocessor for search engines.